1. Field of the Invention
The present invention relates to a method and a system for viewing and enhancing images on a display of a mobile device, which includes the display, memory and a processing means for bit images, an input device for receiving bit images, and in which a bit image is received and processed into a smaller scale using a pre-selected scaling algorithm and enhanced using a enhancing algorithm and opened for processing, in which the enhancing includes one or several following procedures: color and contrast enhancement, sharpening, color management, and dithering.
2. Description of the Prior Art
An imaging mobile device can capture and receive images of various sizes. It is required that these images must be able to be viewed on the display of the device. Typically the display is relatively small, both physically and in number of pixels. Therefore, the display size is often relatively small compared to the image size. The image size must be reduced so that the image fits into the display. This requires downscaling or decimation algorithms. Sometimes only part of the image contains interesting information. Varying level of zooming with panning support is required for showing the details at the area of interest. Zooming can be implemented using upscaling or interpolation algorithms. The downscaling and upscaling algorithms must be of an adequate quality. Otherwise artifacts, such as aliasing effects, jagged edges, excessive smoothing or pixelization, will be introduced to images.
Mobile platforms set strict limits on the amount of memory and processing power available for image processing and enhancement algorithms. Large images consume a great deal of memory and processing power. The amount is directly or exponentially relative to the number of pixels in an image. Therefore it may be impossible to view large images on a mobile device.
Another problem with the current generation of mobile displays is insufficient image quality. Especially when images lack properties that produce good image quality, a lack of the same properties in the display module will produce non-optimal perceived image quality. Typically these features include sharpness, contrast, color contrast, and saturation of images. Quantization artifacts may also be visible, due to the insufficient bit depth of displays. These features can be enhanced with image processing means, but the available processing power and amount of memory may limit or prevent utilization of these methods.
The trivial solution for the low-memory problem is to limit the image size that can be viewed with the device. In that case, some images will not be viewed if there is insufficient memory available. This solution is used in many current products. Though this solution is very simple, it is also clearly very constricting.
The image can be downscaled during opening. Many publications about image resizing and scaling can be found in article and patent databases. Reducing the image size during decoding and opening allows larger images to be opened. Such a solution is used, for example, in the current Nokia® Series60 image viewer. The image is downscaled during decoding, to match the displayed image size as closely as possible. Depending on the encoded image format, scaling can be done sometimes very effectively [U.S. Pat. No. 6,067,384]. However, solutions of this kind may limit the available resizing factors to a few predetermined values. Some formats require standard resizing procedures, which are not able to utilize encoding properties to reduce processing complexity, though these procedures can be still applied at the same time as the image pixels are read from the image source. The same approach can be modified to be suitable also for streaming type input. The drawback of this approach is its inflexibility in supporting various levels of resizing, i.e. downscaling, zooming, and panning. For example, when a larger zoom ratio than the initial opening zoom ratio is wanted, the image must be re-opened. This takes more time, due to the series of re-openings and at some point the system may run out of memory.
The insufficient or poor visibility of images can be enhanced by adjusting the image manually. For example, the user can modify the contrast and saturation of the image. However, this is quite inconvenient, as the adjustments must be made individually for each image. In addition, the user is required to have some experience of image processing. A more advanced solution to the enhancement of the appearance of images is to use automatic image adaptive and display specific enhancements [WO03083775]. For example, histogram-analysis-based contrast and color contrast algorithms can be applied [U.S. Pat. No. 6,148,103]. A proper sharpening algorithm [WO2004/036449A1] too may produce a more satisfactory impression of image quality. Finally, the image appearance on a specific display can be optimized by display-specific compensations and processing, such as color management and dithering [US2003179393]. The algorithms in the enhancement chain can be also modified or combined in effective and robust co-operation, for example, sharpening and contrast enhancement can be combined [EP1242975]. Many references related to individual enhancement algorithms can be found in publication and patent databases.
The main problems related to the prior-art solutions are:
Out of memory: The image to be viewed is too large to fit into the available memory. The system may run out of memory immediately during opening, if downscaling during decoding is not used. Even if downscaling is included in decoding, the amount of memory may be insufficient for re-opening during zooming.
Artifacts due to a deficient downscaling algorithm: Scaling algorithms require that the input image for the algorithm is the best possible. Images already scaled using an incompatible process will not be optimal input for scaling algorithms. Proper downscaling also requires some spatial filtering to be included in the scaling process. The lowest level method, called nearest neighbor, does not include filtering. It senses processing power very effectively, but its lack of filtering produces annoying aliasing artifacts relating to the high-frequency content of images. Despite the use of a filtering downscaling algorithm during decoding, artifacts may occur if the scaling algorithm does not support an accurate scaling ratio to display size. Also downscaling to display size after zooming and re-opening may produce aliasing artifacts, if another re-opening with downscaling is not performed. Another artifact related to scaling is blurring, which occurs if too powerful filtering is used. Both aliasing and blurring are especially harmful if the image is going to be enhanced with a sharpening algorithm. These artifacts may decrease the quality of the sharpening, or prevent its use completely.
Artifacts due to a deficient zooming algorithm: The simplest frequently used zooming algorithm is called pixel copy. In this algorithm the input pixels are repeated, to form a larger image. The pixels of the resulting image are seen as larger squares, instead of as individual pixels. The originally smooth edges also become jagged. A better algorithm with some spatial filtering method is therefore required.
Changes in response time: Image downscaling during opening causes a series of re-openings if the downscaling or zooming ratio is changed. Because the image opening, image reading from file system, and image decoding are very time-consuming processes, the response time of the system increases abruptly when re-opening is performed.
Lack of enhancement or inadequate enhancement quality: Non-optimal construction of the complete processing chain may produce poor enhancement quality or prevent the use of enhancements. For example, image-scaling artifacts may destroy the sharpening operation, if proper scaling algorithms are not used. In addition, processing power resources may prevent the use of complex enhancement algorithms.
Slow operation of enhancements: Processing power may be insufficient for an acceptable processing time, if enhancements are not implemented optimally.
Slow operation or insufficient quality of the entire processing chain: The entire image-processing chain may be non-optimally built for mobile use and the mobile environment. The problems can be solved with the invented method: